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The Crew Exploration Vehicle Parachute Assembly System (CPAS) project conducts 
computer simulations to verify that flight performance requirements on parachute loads and 
terminal rate of descent are met. Design of Experiments (DoE) provides a systematic method 
for variation of simulation input parameters. When implemented and interpreted correctly, 
a DoE study of parachute simulation tools indicates values and combinations of parameters 
that may cause requirement limits to be violated. This paper describes one implementation 
of DoE that is currently being developed by CPAS, explains how DoE results can be 
interpreted, and presents the results of several preliminary studies. The potential uses of 
DoE to validate parachute simulation models and verify requirements are also explored. 
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I. Introduction 

T HE Crew Exploration Vehicle (CEV) Parachute Assembly System (CPAS) is used to recover the Orion vehicle 
after it reenters the atmosphere from Earth orbit. Because the vehicle will be human-rated, strict requirements 
are levied on CPAS parachute flight performance by the National Aeronautics and Space Administration (NASA). 
Parachute loads and terminal rate of descent (ROD) must remain under specified maximum values. 1 To ensure these 
requirements are met, the CPAS project conducts computer simulations of parachute performance using NASA- 
developed 6 Degree-of-Freedom modeling tools: the Decelerator System Simulation (DSS) and Decelerator System 
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Simulation Application (DSSA). DSS and DSSA utilize input parameters that characterize the behavior of 
parachutes. These parameters are varied within specified dispersion ranges, and simulation results are analyzed to 
verify that requirements are met. Design of Experiments (DoE) provides a method to capture the effects of many 
possible combinations of parameters. In turn, DoE can potentially be used by CPAS to validate simulation models 
and verify parachute flight performance requirements. 2 


Table 1. Example 3 3 Full Factorial Study 

Levels: l=minimum value, 2=nominal value, 
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II. DoE Theory 

DoE is a collection of statistical methods that can be used 
to solve a design problem in many industries. Multiple input 
parameters, or factors, are varied between discrete levels and 
the effects on one or more output parameters, or Measures of 
Effectiveness (MoEs), are observed. 3 Conducting a DoE study 
often involves defining and running a complete or full 
factorial experiment, which examines every permutation of 
factors and levels. 4 The number of cases that must be tested in 
a full factorial experiment is determined by the number of 
factors and levels. For example, if 9 factors are varied at 3 
levels, then running a full factorial study requires 3 9 or 19,683 
cases. Table 1 displays an example full factorial matrix for a 
study with 3 levels and 3 factors, for a total of 3 3 or 27 cases. 

By conducting an experiment using a test matrix like the 
one shown in Table 1, the effects on the MoEs from each 
combination of factors will be observed. Once all the 
experiments have been run, the results can be examined using 
Analysis of Variance (ANOVA), a statistical method to 
determine significant interactions between factors and 
combinations of factors. ANOVA calculates ^-values, 
indicating which interactions are significant, as well as mean 
square and F-values that show the relative degree of 
significance for each interaction. In addition, plots produced 
by an ANOVA computer program can be used to graphically 
represent complex interactions. 


III. Implementation of DoE with CPAS Simulation Tools 

DoE is applied to CPAS by conducting a full factorial experiment using parachute simulations. The MoEs 
chosen for the CPAS DoE study are the parachute flight performance parameters: inflation loads and terminal ROD. 
For the preliminary DoE studies, two types of variables were used as factors: parachute inflation factors and 
physical factors. Each of these factors, listed in Table 2, is defined either in Knacke’s parachute design manual 5 or in 
the CPAS model memo. 6 


Table 2. DoE factors used in preliminary CPAS studies 


Factor 

Definition 

n 

Canopy fill constant 

expopen 

Opening profile shape exponent 

C 

D 

Drag coefficient (of full-open parachute) 

c 

k 

Over-inflation factor 

t 

k 

Time to ramp down after stage over-inflation 

£ 

Reefing ratio for inflation stage 

q 

Dynamic pressure 

Weight 

Total vehicle weight 


A. Relating Parachute Inflation Factors to 
the Drag Area Curve 

The first five factors are the parachute inflation 
factors. The values and dispersions of these factors 
are derived from flight test data and are published 
in the CPAS model memo. Each of these factors 
directly affects the drag area ( C D S) curve, shown in 
Fig. 1. Parachute loads are roughly equal to drag 
area multiplied by dynamic pressure; therefore, 
changes in drag area directly affect loads. 

The C D S curve for each parachute stage is made 
up of four components: a) an exponential growth 
segment, b) a linear over-inflation segment, c) a 
ramp down segment, and d) a constant segment. 
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The exponential growth segment (a) of the C D S curve begins at initial time, t t and ends after the elapsed stage fill 
time, tf. Fill time is defined by Eq. (1), where D 0 is the parachute diameter, V t is the initial airspeed, ^ is the initial 
reefing ratio, e i+1 is the final reefing ratio, and n is the canopy fill constant. 


t f =n-D 0 


V^i+l T 


V 


( 1 ) 


Drag area growth for segment (a) as a function of time is given by Eq (2), where (C D S) a is the initial drag area, 
(C D S) b is the final drag area, t is the simulation time, and expopen is the opening profile shape exponent. 


(c D sXt) = (c D s) a+ ((c D s) b -(c D s) a ). 


(t-ti) 


\ expopen 


( 2 ) 


At the end of segment (a), when the drag area first reaches (Q>S%, the C D S curve becomes linear for the duration of 
segment (b) until (C D S) peak is reached. The infinite mass over-inflation factor, Q, defined in Eq. (3) determines the 
value of (C D S) peak . 

(C D S) p „ t 

(C D S)„ (3) 


C,= 


As soon as the peak value is reached, segment (c) begins, and the drag area decreases according to the ramp down 
function given by Eq. (4), where tf p is the elapsed time to peak drag area and t k is the time to ramp down after stage 
over-inflation. Eq. (4) is valid until a time t k has elapsed, at which point segment (c) ends and the C D S curve 
becomes constant for the duration of segment (d) until the next stage begins. 


(c D sXt)=(c D s) I ^.(c 1 ) ( ‘* 


( 4 ) 


In CPAS simulations, Drogues and Pilots are modeled with infinite mass inflation (including over- inflation) and 
Mains are modeled with finite mass inflation. For finite mass inflation, C k is equal to 1 and there is no over- inflation. 
As a result, values of C k and t k are not reported for CPAS Mains. 



tj Time,t 


Figure 1. Drag area curve for a single parachute disreefing stage 6 

t t = initial time; tf= stage fill time; tf p = stage time to peak drag area; 

t k = time to ramp down after stage over- inflation; (C D S) a = initial drag area; 

(C D S) peak = peak drag area; (C D S) b = final drag area 
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B. Conducting CPAS DoE Studies in MCDS 

For the initial full factorial studies described in this paper, each factor was varied to either two or three levels. 
Each level corresponds to the minimum, maximum, and nominal dispersion values from the model memo. It is 
possible to conduct DoE studies with more levels, providing a higher resolution of results, but the number of runs 
increases rapidly and is limited by available memory and processing power. 

DoE is implemented in MATLAB by a modified version of the CPAS Monte Carlo Decelerator Simulation 
(MCDS), 7 a tool used to run and display output from multiple parachute simulation runs. First, MCDS sets up all the 
necessary parameters and options to run the chosen parachute simulation (usually DSS or DSSA). In the process, it 
creates a MATLAB data structure containing the values of each factor at each level, as well as default values for 
other input parameters. Next, a full factorial matrix, similar to Table 1, is produced using the MATLAB Statistics 
Toolbox. MCDS then runs through each row of the matrix, choosing factor values for each case according to the 
content of that row. Once all the cases have been run, MCDS opens an interactive plotting utility that displays the 
output data. 


IV. Interpreting DoE Results 

After a DoE study has been run, the results are interpreted in several ways. Any cases exhibiting significant or 
interesting effects are isolated to be examined more closely. For example, if DoE is being used to verify the 
requirements on parachute flight performance parameters, then results outside of the requirements need to be 
investigated and explained. This often involves additional DoE studies focusing on a tighter dispersion of factors 
around the suspect cases. These studies may reveal potential data gaps that can be filled through additional hardware 
or flight tests. 

In addition, ANOVA can also be used to identify factors and combinations that cause large variations in MoE 
values, compare degrees of significance, and plot complex interaction effects. ANOVA provides results that can 
help determine if parachute parameters currently modeled independently are actually interrelated. Quantifying the 
effects of input parameters on parachute performance through DoE provides statistical grounds to validate parachute 
models and justifies the use of those models to verify requirements. 

V. Preliminary DoE Studies in DSSA 

The preliminary DoE studies discussed in this section were built from the DSSA case file for a CPAS Cluster 
Development Test (CDT-2-3). The factors used in this DoE study are unique for each parachute stage, but were 
varied only for the first stage of the Drogue parachutes. Factors for all subsequent Drogue and Main parachute 
stages were held constant at their nominal values. 

A. 2 8 study (256 cases) 

The first study was conducted to establish architecture for DoE and investigate the usefulness of DoE results. 
This study consisted of 2 levels and 8 factors for a total of 2 8 or 256 cases. MCDS took about 35 minutes to execute 
and process 256 DSSA runs. The 8 factors included 5 parachute inflation parameters (n, expopen, C D , C h and t k ) and 
3 physical parameters (s, q , and Weight). The physical parameters were held constant in this study because they do 
not have well-defined dispersions, but they can easily be varied in the future. They were included to provide a more 
complete picture of all the factors affecting parachute performance. Because the physical parameters are held 
constant, this particular study only shows the effects of the 5 parachute inflation parameters. Therefore, only the first 
2 5 or 32 cases are unique; the rest of the 256 cases are duplicates and are not included in the results. 

Several observations were made from the results of this study. First, CPAS Analysis Engineers had intuited that 
ROD is driven by C D , which this study confirmed. The statistics plot shown in Fig. 2 indicates that ROD varies by 
almost exactly ±o. The MDCS statistics plotting utility allows the user to select a specific point and determine the 
value of each factor for the case that point represents. Such an investigation reveals that all the cases clustered 
around +g have a high C D and all the cases clustered around -o have a low C D . This indicates that the other inflation 
parameters likely do not have much effect on ROD. ROD was examined in the follow-up 3 5 study as well, but the 
results were almost identical. It was decided that DoE studies should focus on parachute inflation loads for the time 
being, and a more detailed study of ROD could be conducted at a later time. It should be noted that the C D values 
reported in the model memo are best fits from test reconstructions of ROD and do not necessarily match the actual 
steady-state C D of the parachutes, because ROD is driven by cluster dynamics. 8 
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Figure 2. Statistics Plot for Terminal ROD from 2 8 study, zoomed in to show the 32 unique cases 

The effects on peak load for the first stage of the Drogue parachutes were also examined. The variation in peak 
loads is shown in Fig. 3 on a plot of Drogue riser load versus time. The highest loads (~ +2 g) occurred when C D , 
expopen, and C k were at their maximum values and n was at its minimum value. This indicates that significant 
interaction in loads probably exists between three inflation parameters: n, expopen, and C k . It also revealed that 
some factors do not significantly affect peak loads, but a more thorough analysis with ANOVA was required to 
obtain higher fidelity results. 
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Figure 3. Load traces from 2 8 study with peak loads for Drogue stage 1 indicated by dots. Due to 

memory constraints, only the first 20 load traces are plotted here, but all peak loads are shown. 
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B. 3 5 study (243 cases) 

A follow-up study was conducted using three levels (maximum, minimum, and nominal) to improve the fidelity 
of results. To keep processing time down, the physical parameters were removed, leaving only the 5 parachute 
inflation parameters as factors. With 3 levels and 5 factors, this study encompassed 3 5 or 243 cases, and took about 
30 minutes to execute and process in MCDS. As shown in Fig. 4, the peak load distribution was wider, and the 
peaks did not overlap as much as in the two level study. Also, the same factors influenced peak loads as in the 2 8 
study; the highest loads were at the same values, but they were farther from the mean for the 3 5 study. 


DS5A DoE {243 Cycles): CDT-2-3 
Chute Load vs. Time - Ramp Clear 



Figure 4. Load traces from 3 5 study with peak loads for Drogue stage 1 indicated by dots. Due to 

memory constraints, only the first 20 load traces are plotted here, but all peak loads are shown. 


Table 3. ANOVA results from 3 5 study 


Analysis of Variance 


SOUECfr 

Sun Sq. 

d. £. 

Heart Sq. 

¥ 

Pr&&>-F 


Cd 

90226931 . 4 

2 

45113465.7 

420 . 39 

0 


n 

1602961672 

2 

801480786 

7468.68 

0 


expopen 

646971496 

2 

323485748 

3014.44 

0 


€k 

1380350966 3 

2 

690175483.2 

6431.47 

0 


tik 

2251240. 2 

2 

1I25C20. 1 

10.49 

0. 0001 


T3 r iT““ ---- 


T 


— 



Cd*expopen 

759447 . 3 

4 

189861.8 

1.77 

0. 13991 


Cd*Ck 

589214. 1 

4 

147303.5 

1.37 

0. 2479® 


Cd*tk 

35452 . 4 

4 

8863.1 

0.08 

0. 9876 


n*exp»pen 

65481247.7 

4 

16370311.9 

152.55 

0 


n*Ck 

66482900.2 

4 

16620725.1 

154.08 

0 



3091174. 9 

4 

772793. 7 

7.2 

0 



4 

4 

i*m?n-7n i 

171 7K 

ft 


expopen*tk 

2864367.1 

4 

716091.8 

6.67 

0.0001 


Ck*tk 

_2£312Q7.8 _ 

4 

7Q7.801 . 9 




Cd*n*expopen 

159288.1 

8 

19911 



0.19 

0. 9925| 


Cd*n*Ck 

105136 

8 

13142 

0.12 

0. 9982* 


Cd*n*tk 

41245.3 

8 

5155.7 

0.05 

0.9999| 


Cd*expopen*Ck 

71548.5 

8 

8943.6 

0.08 

0. 9996| 


Cd*expopen*ck 

391106 

8 

4888.8 

Q-05 

4 ! 



JJ5&92 

a__ 

At 62^L_ AJ^A 


n*exp»pen*Ck 

3195994 9 

a 

399499 4 

3.72 

0. 0007 


n*exp»pen*fek 

6297571 . 9 

8 

787196 . 5 

7.34 

0 


n*Ck*tk 

5483887.2 

8 

685485.9 

6.39 

0 


expcpen*Ck*tk 

5640867. 8 

8 

705108.5 

6.57 

0 


Error 

12126280. 1 

113 

107312.2 




Total 

3950321453.9 

243 





constrained (type m) suras ot squares 


6 


ANOVA was applied to this study to 
obtain more detailed results. The ANOVA 
results, shown in Table 3, highlight the 
factors and combinations that have a large 
effect on peak load variation. Low ^-values 
(Prob>F), below an arbitrary probability of 
a=0.05, indicate that an interaction is 
statistically significant. Interactions with 
higher mean square and F-values are more 
significant, even if they have the same 
p- values. 

The interactions boxed by dashed lines 
in Table 3 are statistically insignificant 
because they have ^-values higher than 
0.05. These are all combinations involving 
Co, indicating that there are no significant 
interactions between drag coefficient and 
any other factor. The factors and 
interactions boxed by solid lines in Table 3 
are the most significant. They all have p- 
values of zero and very high mean square 
values. According to this table, the three 
most significant factors, both alone and in 
combination, are n, expopen, and C k . The 
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unboxed factors and combinations are only marginally significant. From these results, it can be concluded that loads 
are fundamentally a function of three parachute inflation parameters: n, expopen, and C k . First order interactions 
between these three are also significant, but no two or three way interactions have as much of a contribution as the 
individual factors. 


Table 4. ANOVA results from 3 3 study 
Analysis of Variance 


Source 
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Constrained (Type III) sums of squares. 


C. 3 study (27 cases) 

The next step involved isolating only the 
three most significant parameters. Again, 3 
factors were used with 3 levels each, resulting 
in 3 3 or 27 cases. The main goal of this study 
was to examine the significant interactions 
between parameters. An ANOVA analysis, 
shown in Table 4, revealed that all individual 
factors and first order interactions were 
statistically significant, with individual 

factors much more significant than the 
combinations. The most significant factor was 
n , followed by C k and expopen. 

Main effects plots reveal the peak load trends for each of 
these factors. Fig. 5 shows that peak loads are inversely 
proportional to n and directly proportional to expopen and C k . 

The multi-vari plots, shown in Fig. 6, indicate that these 
trends generally hold even when considering complex 
interactions between the three factors. One interesting 
anomaly is denoted by the circled cluster of points in Fig. 6a. 

At this particular combination of two factors (high n and low 
C k ), the peak load trend in the third factor, expopen is not 
completely consistent with all the other cases. This indicates 
that some combinations of two factors can affect peak load 
trends in the third factor and that one or more of the 
dispersed parameter values may need to be reevaluated. 

Though seemingly insignificant, this effect warrants closer 

examination, and the next study investigates it in more detail. Figure 5. Main effects plot from 3 3 study 





Figure 6. Multi-vari plots from 3 3 study. Each plot has a 
constant C k with n varied along dashed lines and expopen 
varied along solid lines. 


D. 5 3 study (125 cases) 

For the final study, the dispersion range on each 
of the three most significant factors was doubled to 
see if peak load trends hold or break down for a 
wider dataset outside the range of flight test 
experience. The same 3 factors were used as in the 
previous study, but with 5 levels instead of three for 
a total of 5 3 or 125 cases. The middle three levels 
were the same as the original three, reflecting the 
model memo dispersions for each factor, but the 
outer two levels encompassed an extension of the 
dispersion ranges. To calculate the “double 
dispersions,” the difference between each model 
memo nominal and dispersed value was doubled. 
For example, the nominal n value was 3.7 and the 
model memo dispersion minimum was 2.25, so the 
double dispersion minimum is 0.8. Because the 
model memo dispersions are not necessarily 
equally distributed about the nominal value, neither 
are the doubly dispersed values. In other words, the 
minimum dispersed value might be further from the 
nominal than the maximum dispersed value, or vice 
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versa. The first significant result is that outside of the model memo dispersion range, C k has a larger effect on peak 
loads than n. In addition, as shown in Fig. 7, peak load trends still hold for individual factors. 


* 1C* 



Figure 7. Main effects plot from 5 3 study 


However, many peak load trends do not hold for complex interactions outside of the model memo dispersion 
range. These trend breakdowns can be seen dramatically in Fig. 8 as tight clusters of points and large zigzags. The 
tight clusters of points in Fig. 8a and 8b indicate that there is little variation in peak loads as one factor changes 
value. In contrast, the large zigzags in Fig. 8c, 8d, and 8e indicate large discrepancies in peak load values as one 
factor changes. These results actually help to validate the simulation model because they show that the effect of 
complex interactions outside the model memo dispersion range do not follow predictable trends. Part of the reason 
for these results is that some factor values outside the dispersion range are not realistic. For example, any values of 
C k less than 1, although included in this study as shown in Fig. 8a, are not physically possible. The breakdown of 
trends outside the standard dispersion range suggests that the dispersion limits have been set at appropriate values. 



O expopen = 0 5G Q gKpnpBn = 1 .OB 0 = 1.6 ^ expopsn = 1.36 ft fixpopan = 2.35 1 

Figure 8. Multi-vari plots from 5 3 study. Each plot has a constant C k with n varied along dashed lines 
and expopen varied along solid lines. 
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VI. Conclusion 

The results of the preliminary DoE studies presented in this paper raise some interesting questions about the way 
parachute inflation is currently modeled. First, the results indicate that the value of C k may not be completely 
independent, but rather a complex function of several factors. If this is true, it may be not be advantageous to 
continue dispersing values for C k separately from the other factors that drive it. In addition, the breakdown in peak 
load trends for certain combinations of factors indicates that some combinations of values within the model memo 
dispersion range may not be physically possible. One possible remedy is to bound the drag area curve in addition to 
dispersing each parameter individually. If any combination of parameters results in a drag area curve that does not 
correspond to a possible physical parachute inflation, that case and its results would be ignored. 

There is significant forward work still to be done on the CPAS DoE study. First, the preliminary DSSA studies 
described in the previous section need to be rerun in DSS, because it is the primary simulation tool used to verify 
requirements. Second, the studies described in this paper only varied parameters for the first stage of the Drogue 
parachutes and need to be extended to subsequent Drogue and Main stages. In addition, physical parameters such as 
reefing ratio, weight, and initial q should be dispersed. Finally, CPAS requirements must be verified using 
benchmark cases representative of integrated CEV flight reentry and abort trajectories. These benchmark cases need 
to be developed and studied so CPAS can see how well the requirements are met. 

DoE helps perform two main functions for the CPAS project: validation of simulation models and verification of 
parachute flight performance requirements. Results from DoE studies provide statistical measures of the effects of 
input parameters on parachute performance. DoE also offers a unique perspective from Monte Carlo simulations 
because it accounts for interactions between input parameters rather than varying each parameter randomly. If CEV 
trajectories and parachute performance can be accurately modeled, DoE has the potential to aid in validating model 
memo parameter dispersions and verifying that requirements will not be violated. 
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